In recent years, techniques for recording still images or moving images (video) of a three-dimensional space have been paid attention, and research is underway. However, there are various problems to be solved in order to encode and decode all information of a photographed (captured) three-dimensional space.
Thus, there have been proposed a minute viewpoint movement-adapted image that reduces technical barriers by limiting a viewpoint movement range to, for example, a typical TV-viewing range (a minute viewpoint movement range: a range where a viewer merely moves his or her head (performs a parallel movement), as well as image encoding and decoding techniques.
As described above, although various approaches have been proposed as techniques for encoding and decoding three-dimensional images, there are problems with the amount of calculation and the amount of data in encoding and decoding three-dimensional images even when assuming a minute viewpoint movement.
For example, in a proposed technique, cameras are located at the center, left, and right and, based on still images acquired from the center, left, and right cameras and a depth map thereof, there are recorded images and depths of only objects not included in a front image (a center image) among the center image and left and right viewpoints.
Then, when decoding, pixel shift according to each depth is performed by geometry calculation according to the amount of viewpoint movement of a viewer. Portions with a lack of pixel data due to the shift are interpolated using encoded data of the left and right images.
However, while this technique can, for example, reduce the amount of decoding data, geometry calculation is performed on encoded three-dimensional image information, thus resulting in an enormous amount of calculation.
In addition, for example, there has been proposed a technique in which a lens array and light collecting lenses are arranged on the entire surface of a high-precision camera sensor to encode all of moving images from respective viewpoint positions. When decoding, viewpoint movement may be made by performing a reverse procedure to a procedure at the time of encoding.
However, this technique does not use, for example, geometry calculation. Accordingly, while the technique may reduce the amount of calculation, it encodes a plurality of viewpoint images, thus using an enormous amount of encoding data.
As described above, even when specialized to a minute viewpoint movement range, although it is possible to reduce the amount of calculation and the amount of data as compared to encoding of all pieces of three-dimensional image information, it is still insufficient and there is imposed a very large load.
By the way, in the past, various approaches have conventionally been made for techniques for recording still images or moving images of three-dimensional spaces, i.e., techniques for encoding and decoding three-dimensional images.
Patent Document 1: Japanese Laid-open Patent Publication No. 2013-046263
Patent Document 2: Japanese Laid-open Patent Publication No. H09(1997)-027969
Patent Document 3: Japanese Laid-open Patent Publication No. 2009-251141
Patent Document 4: Japanese Laid-open Patent Publication No. 2001-256482
Patent Document 5: Japanese Laid-open Patent Publication No. 2004-152133
Patent Document 6: Japanese Laid-open Patent Publication No. 2007-334583
Non-Patent Document 1: Masayuki Tanimoto, “International Standard Technology for Practical Use of Free-viewpoint Television,” ICT Innovation Forum 2013, Oct. 1, 2013
Non-Patent Document 2: Tehrani et al., “Use Cases and Requirements on Free-viewpoint Television (FTV),” ISO/IEC JTC1/SC29/WG11, MPEG2013/N14104, Oct. 2013
Non-Patent Document 3: Arai et al., “Integral three-dimensional television with video system using pixel-offset method,” OPTICS EXPRESS, Vol. 21, No. 3, pp. 3474-3485, Feb. 2013